The logic of adaptive behavior : knowledge representation and algorithms for the Markov decision process framework in first-order domains

Learning and reasoning in large, structured, probabilistic worlds is at the heart of artificial intelligence. Markov decision processes have become the de facto standard in modeling and solving sequential decision making problems under uncertainty. Many efficient reinforcement learning and dynamic programming techniques exist that can solve such problems.\ud Until recently, the representational state-of-the-art in this field was based on propositional representations.\ud \ud However, it is hard to imagine a truly general, intelligent system that does not conceive of the world in terms of objects and their properties and relations to other objects. To this end, this book studies lifting Markov decision processes, reinforcement learning and dynamic programming to the first-order (or, relational) setting. Based on an extensive analysis of propositional representations and techniques, a methodological translation is constructed from the propositional to the relational setting. Furthermore, this book provides a thorough and complete description of the state-of-the-art, it surveys vital, related historical developments and it contains extensive descriptions of several new model-free and model-based solution techniques

Relational Representations in Reinforcement Learning: Review and Open Problems

This paper is about representation in RL.We discuss some of the concepts in representation and generalization in reinforcement learning and argue for higher-order representations, instead of the commonly used propositional representations. The paper contains a small review of current reinforcement learning systems using higher-order representations, followed by a brief discussion. The paper ends with research directions and open problems.\u

A survey of reinforcement learning in relational domains

Reinforcement learning has developed into a primary approach for learning control strategies for autonomous agents. However, most of the work has focused on the algorithmic aspect, i.e. various ways of computing value functions and policies. Usually the representational aspects were limited to the use of attribute-value or propositional languages to describe states, actions etc. A recent direction - under the general name of relational reinforcement learning - is concerned with upgrading the representation of reinforcement learning methods to the first-order case, being able to speak, reason and learn about objects and relations between objects. This survey aims at presenting an introduction to this new field, starting from the classical reinforcement learning framework. We will describe the main motivations and challenges, and give a comprehensive survey of methods that have been proposed in the literature. The aim is to give a complete survey of the available literature, of the underlying motivations and of the implications if the new methods for learning in large, relational and probabilistic environments

A Reinforcement Learning Agent for Minutiae Extraction from Fingerprints

In this paper we show that reinforcement learning can be used for minutiae detection in fingerprint matching. Minutiae are characteristic features of fingerprints that determine their uniqueness. Classical approaches use a series of image processing steps for this task, but lack robustness because they are highly sensitive to noise and image quality. We propose a more robust approach, in which an autonomous agent walks around in the fingerprint and learns how to follow ridges in the fingerprint and how to recognize minutiae. The agent is situated in the environment, the fingerprint, and uses reinforcement learning to obtain an optimal policy. Multi-layer perceptrons are used for overcoming the difficulties of the large state space. By choosing the right reward structure and learning environment, the agent is able to learn the task. One of the main difficulties is that the goal states are not easily specified, for they are part of the learning task as well. That is, the recognition of minutiae has to be learned in addition to learning how to walk over the ridges in the fingerprint. Results of successful first experiments are presented